Papers with knowledge-grounded dialogue
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark (2022.tacl-1)
Copied to clipboard
| Challenge: | Knowledge-grounded dialogue systems powered by large language models often generate responses that, while fluent, are not attributable to a relevant source of information. |
| Approach: | They propose to evaluate the validity of 12k dialogue turns generated by neural dialogue systems trained on three knowledge-grounded dialogue corpora and to use them to analyze eight evaluation metrics. |
| Outcome: | The proposed evaluation metrics rely on spurious correlations, do not reliably distinguish attributable abstractive responses from unattributable ones, and perform substantially worse when the knowledge source is longer. |
CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems (2024.findings-acl)
Copied to clipboard
Abbas Ghaddar, David Alfonso-Hermelo, Philippe Langlais, Mehdi Rezagholizadeh, Boxing Chen, Prasanna Parthasarathi
| Challenge: | CHARP is a testbed for knowledge-grounded dialogue evaluation of models trained on FaithDial data. |
| Approach: | They propose a testbed for evaluating models trained on FaithDial with annotation artifacts that may bias models towards completely ignoring the conversation history. |
| Outcome: | The proposed model fails to accurately evaluate the conversational history and lacks hallucination detection. |
Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue models lack prior and posterior knowledge selection . prior selection module may not learn to select knowledge properly because of lack of posterior information . |
| Approach: | They propose a knowledge distillation-based training strategy to remove the exposure bias of knowledge selection. |
| Outcome: | The proposed model improves on two knowledge-grounded dialogue datasets. |
Enhancing Knowledge Retrieval with Topic Modeling for Knowledge-Grounded Dialogue (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to knowledge retrieval are limited by the knowledge base encoder, but our work focuses on the knowledge-base encoder. |
| Approach: | They propose an approach that utilizes topic modeling on the knowledge base to improve retrieval accuracy and as a result, improve response generation. |
| Outcome: | The proposed approach can improve retrieval and generation performance on two datasets. |
Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation methods for factual consistency in knowledge-grounded dialogues are unreliable and limit their applicability. |
| Approach: | They propose an automatic evaluation metric for factual consistency in knowledge-grounded dialogue using automatic question generation and question answering. |
| Outcome: | The proposed evaluation metric consistently shows higher correlation with human judgements. |